RooJSolutions

PHP just does some things better. cloud backups, pecl-expect

2013-03-04 00:00:00

Article originally from rooJSolutions blog
Backups, yes we do backup occasionally, and I've been looking for a better solutions than my historical office<->hosting replication, which although cheap, always made me wonder if I was still making copies. So after our recent office move, and minor server upgrade, I thought better double check on it all.. As usual, the thing had failed due to various reasons, and needed replacing.

So since I've been thinking about a new solution for our clients, I decided to go ahead and try it out for our data. I've been googling affordable cloud based storage for a while and found a company onlinestoragesolutions.com, the pricing is very reasonable US$45 for 2 years currently, with unlimited everything. There are a few review sites that seem to throw some cold water over the offer, but I had one client sign up without any major issues (for another reason), so at that price I though let's give it a go....

One of the key features they advertise is rsync, which since we run all linux machines would be ideal. However after paying, and getting access, I realized it's not quite a simple as pointing your rsync at their server. You need to set up an ssh tunnel to route rsync through.

Can't be that difficult I thought.... Turns out that setting up a password based ssh tunnel, automatically on a cron job, is no small task, there are questions all over stackexchange and various forums, none that I saw managed to find a solution. Most of the suggestions are based around the 'expect' program, a usefull unix tool which can be used to script ssh access. The problem in this case was that setting up the tunnel, then doing the rsync, and then closing the tunnel is not something that a bash, expect or any other method I found could do easily if at all.

So almost at the point of giving up, I started looking around at php's popen (which would not work either), and fell over the pecl expect extension.

Below is the result of a few minutes coding, which does exactly what is needed, and can be run directly from cron. Feel free to escape from overpriced backup solutions..

Interesting Problems.

2012-11-02 00:00:00

Article originally from rooJSolutions blog

If there is one reason I would put down to why I still enjoy coding after all these years, it is the pleasure finding a really challenging problem and being able to solve it quickly and efficiently.

This week saw one of those problems, strange, obtuse, and to most non crazy people would seem to be an odd thing to get excited about. It all started with going through the requirements list on one of our projects.

As I've mentioned before, we are building quite heavily ontop of Xtuple, a Web based GUI using the RooJS Toolkit, most of that part has been just process orientated - add this feature, and it just works. But some of the implementation process has been a challenge.

The company we are working with has 2 main offices (and more comming on line soon), and we have successfully migrated them off of netsuite, which in reality made an absolute mess of their accounts. When migrating to Xtuple (which in essence is a postgresql database, with a seemingly endless number of table triggers, stored procedures and views), we concluded quite early on that setting up a seperate instance for each office would be the best way to go.

Core issues like difference currencies in each office, so the chart of accounts can be reported each year in the local currency to the tax department and auditors, along with the different end-of-year tax reporting made this a simple decission. The otehr issues was that the basic installation of Xtuple does not really support Multiple companies in the Desktop UI (you have to pay for that feature).

So as we are now live, and just about finished with the rollout issues, we are now on to the more critical things for long term operations - Management reports (and a nice dashboard).

The issue that raised it's head was that the company needs to see consolidated Balance sheet and Income statements. It did not take to much time to call the internal methods in Xtuple 'financialreport()' and in PHP convert this to JSON so that a master script can call the report in both systems, and merge them together.

The snag came later when we realized that the resulting report was a little off.. Due to the magic of differeing accounting periods.

Migrating off Netsuite - The hidden cost of Clouds..

2012-04-28 00:00:00

Article originally from rooJSolutions blog
One of the more interesting projects I have ongoing is a migration and deployment of an ERP system. For a bit of background, our customer was persuaded about 3-4 years ago that moving their accounting and Stock management into Netsuite would be a good idea.

For those who have not heard of Netsuite, it's a Cloud based ERP system, that from my understanding runs on an oracle backend and is totally web based. All the users log into the website, enter the various stock activity ( Purchase Orders, Item Receipt, Bills, Sales orders, Invoices, Item Fulfilments and Various Credit Memos), along with the standard accounting features like Journal entries. etc.... The system keeps track of your stock and should give you a value of stock on hand, along with doing your audit-able reports.

Well, I'm guessing anyone who has worked with ERP systems, probably knows that the devil is in the details for this. The company I have been working with have been using this for over 2 years now, and are beginning to see the ugly side of cloud based system (which is why we are trying to migrate off of it.), Let's have a look through the issues.

Browser rending weirdness OSX and firefox

2011-01-26 00:00:00

Article originally from rooJSolutions blog
Sometimes rendering bugs can be very annoying, IE has it's share of misbehaving, but usually firefox, chrome are consistant and never need much testing. However on one site, prior to going live someone testing on OSX kept saying that the menu bar was broken in Firefox.

Testing on Windows and Linux it looked fine. however on their browser, the horizontal line of buttons for the menu broke into two lines. It was not until I finally did remote control on her PC was I able to see what was going on.

Basically, on first load it rendered fine, but as soon as you went back to the page, the buttons would break. Inspecting it in firebug indicated that the second time it rendered the DOM tree was actually different to the HTML source.

The button code looks a bit like this

<a href="somurl"><div class="btn"><div class="btn-right"><div class="btn-left"><div 
   class="btn-body">Title here</div></div></div></div></a>

The idea is that the div's provide the rounded button edges, and allow nice hover effects all in CSS.

however inspecting the reloaded page indicated the tree looked like

<a href="somurl"></a>
<div class="btn"><div class="btn-right"><div class="btn-left"><div
         class="btn-body">Title here</div></div></div></div>

Where the a tag had broken away from the div, strangely only on one button.

It was only after I ran it through the w3 validator and got an error message about block elements inside and inline one, that I had the idea to rewrite that HTML to use span's, rather than div's. Along with adding display:block to all the CSS for the button elements.

The resulting HTML being

<a href="somurl"><span class="btn"><span class="btn-right"><span class="btn-left"><span 
        class="btn-body">Title here</span></span></span></span>

And amazingly enough it rendered perfectly on all browser.. strange but true....

Leadership lessons, or are MBA courses any good?

2010-10-10 00:00:00

Article originally from rooJSolutions blog

And now for something different.. I have a friend doing a course at HKU on Leadership, and with the feedback I'm getting, the course might just be missing the plot a bit.

From my side of the fence, ever since I gave up on my Masters, I have been generally dismissive of post-graduate courses at Universities. There was a serious sense of 'If you can't do it, teach it..' (or now if you really can't do it, blog about it..). My sense is that there is a reasonably large body of acedemia that suffers from the fact that they are kept on board as a result of quantity of research publications. They however they do not appear to understand the principles behind teaching (which teacher training wonderfully encapsulates). I wonder if tutors at university should be required to undertake a teacher training course, rather than slowly migrating from research into tutoring.

Along with all this, I wonder if the type of students these course attract are the exact opposite to 'self-learners', and are more used to a spoon feed education, especially here in asia.

Anyway back to the subject at hand. HKU was teaching a course based around a Harvard case study about a climb of mount Everest resulting in a number of deaths. The students are given the course material and are asked to produce a short piece explaining it and illustrating their ability to extract the leadership lessons that caused the failure.

This is where adult reading (not the picture kind) comes in handy. In Malcom Gladwell's book Outliers, a chapter is dedicated to an interesting study of plane crashes, and pilot assertiveness. If you read the book in the wrong way it appears to sensensionalize the idea that different countries and cultures have different levels of assertiveness which lead to greater crashes with pilot's of different countries. While this inference tends to lead to great press and I presume book sales. It is not really the message that come accross if you read it in detail.

In essense the book points out that individuals (and perhaps to some degree culturally based) are less assertive, in that they feel unable due to their role in society of workforce to assert their opinions. In the case of the pilot's it was the quiet mention that there might be a small issue with the weather. In the case of the climb it was more a case of Doctor's not asserting the seriousness of health issues, or the sherpa's inability to express that they where concern with various issues about the trip. All these would have likely been non-fatal if both communication had been clear and strong, and the subordinates had felt comfortable in the life threatening situations in overriding the leaders actions.

From the sense I got, the course was being taught on leadership, but unfortunatly learning that there are problems occuring with these type of situations has very little to do with leadership, and more to do with realizing that assertiveness training is an essential part of senior management, and more essential in situations which may be life threatening.

What is worse is that the course dynamic's of putting groups into teams and tasking them with these studies looks like it is a perfect example of why failure happens. Rather than using the course to entrench this positive assertiveness it basically leaves the groups to their own devices to produce a result and does not monitor their interaction trying to illustrate the individual's in the groups own abilility to assert their opinions. Would this type of couse not be better taught better with some kind of situation based role playing and instant feedback.

Company rename and RooJS sites now online

2010-08-25 00:00:00

Article originally from rooJSolutions blog
After what seems like forever in design and migration, I've turned on the roojs.com and roojs.org domains, I'm slowly in the process phasing out my old company name - AK BK Consulting, which is now rooJSolutions

Quite a few of the key pages are missing content at present, but the blog and it's features are mostly moved over.

After I fill out all the missing content, I'll get back to blogging about how most of the Javascript tools and associated backend libraries all work together. Exciting times for a quality platform neutral Javascript Rich User Interface, guaranteed $(wtf) free...

nested trees in mysql - handy stored procedures.

2009-07-03 14:52:00

Article originally from rooJSolutions blog
Storing tree data in Mysql databases, is relatively common, however, all the existing documentation about doing this, makes the whole process rather complex.

If you google it, you will probably find the quite definitive answer at mysql.com, describing the classic parent_id method, and the left/right numbering process. Both of these methods involve rather complex SQL to fetch and update the tree.

In seeking a better solutions for a tree that was infrequently updated, but frequently queried, I thought I'd try seeing if I could write a few stored procedures to simplify the process.

Our basic database structure looks like this:

CREATE TABLE _TREE_ (
    id int(11) NOT NULL auto_increment,
    parent_id   int(11)  NOT NULL DEFAULT 0,
    seqid int(11)  NOT NULL DEFAULT 0,
    depth int(11)  NOT NULL DEFAULT 0,
    leaf int(1)  NOT NULL DEFAULT 0,
    name varchar(128) default '',
    fullpath TEXT default '',
    PRIMARY KEY  (`id`),
    INDEX qlookup( parent_id , seqid , depth)
);

our key components are

id (the nodes id)
parent_id (the nodes parent - pretty clasic)
name (the textual name of the node)
seqid - the generated order item for the whole tree
depth - how deep the node is (usefull for indenting)
leaf - is it a leaf node (eg. has no children) - usefull for icons

The trick of the stored procedures is to correctly generate the seqid, this consists of two parts, the top level wrapper and the recursive prodedure to update the table.


DROP PROCEDURE IF EXISTS _TREE__resequence;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence(i_sep VARCHAR(4))   DETERMINISTIC
  BEGIN
        DECLARE v_p, v_d, v_s INT(11);
        DECLARE v_fp TEXT;
        SET v_fp = '';
        SET v_p =0;
        SET v_d =0;
        SET v_s =0;
        SET max_sp_recursion_depth=255;
        CALL _TREE__resequence_sub(v_p, v_d, v_fp, i_sep, v_s);
        
        
    END $$
DELIMITER ;

   



DROP PROCEDURE IF EXISTS _TREE__resequence_sub;

DELIMITER $$
CREATE PROCEDURE _TREE__resequence_sub(
    i_parent INT(11),
    i_depth INT(11),
    i_fullpath TEXT,
    i_sep VARCHAR(4),
    INOUT i_seqid INT(11)
    )   DETERMINISTIC
    BEGIN
        DECLARE v_nid, v_ex_seqid  INT(11);
        DECLARE v_name VARCHAR(128);
        DECLARE v_leaf INT(1);
        DECLARE v_fullpath TEXT;
        DECLARE done INT DEFAULT 0;

        DECLARE qry CURSOR FOR SELECT id, seqid, name  FROM _TREE_ 
              WHERE parent_id = i_parent ORDER BY seqid;
        DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
        OPEN qry;
        
        REPEAT
            FETCH qry INTO v_nid, v_ex_seqid, v_name;
            
            IF NOT done THEN
                IF v_ex_seqid != i_seqid THEN 
                    UPDATE _TREE_ SET seqid = i_seqid, depth=i_depth WHERE id=v_nid;
                END IF;
                IF i_depth > 0 THEN 
                    SET v_fullpath = CONCAT(i_fullpath, i_sep, v_name);
                ELSE 
                    SET v_fullpath = v_name;
                END IF;
                SET v_leaf =0;
                SELECT COUNT(id) INTO v_leaf FROM _TREE_ where parent_id = v_nid;
                
                UPDATE _TREE_ SET 
                    fullpath = v_fullpath,
                    leaf = IF (v_leaf > 0, 0 , 1)
                    WHERE id=v_nid;
                
                SET i_seqid = i_seqid +1;
                #// do the children..
                CALL _TREE__resequence_sub(v_nid, i_depth+1, v_fullpath, i_sep, i_seqid);
            END IF;
        UNTIL done END REPEAT;
        
        CLOSE qry;
    
    END $$
DELIMITER ;

This does the hard work of iterating through the tree, and updating the sequence number, depth, leaf field and filling in the fullpath field

Now our simple add node code just adds the node in the correct place, bumps the seqid along, so that you can then regenerate the tree.


DROP FUNCTION IF EXISTS _TREE__add_node;

DELIMITER $$
CREATE FUNCTION _TREE__add_node(
    i_parent INT(11),
    i_after INT(11),
    i_name VARCHAR(128)
    ) RETURNS INT(11) DETERMINISTIC
    BEGIN
        
        DECLARE v_depth INT(11);
        DECLARE v_seqid INT(11);
        DECLARE v_ret INT(11);
        DECLARE v_tmp INT(11);
         
        SET v_depth = 0;
        SET v_seqid = 0;
        SET v_ret = 0;
        SET v_tmp = 0;
         
        
        #// grap parent depth.
        SELECT depth +1, seqid + 1 INTO v_depth, v_seqid FROM _TREE_ WHERE  id= i_parent LIMIT 1;
        #// grab previous id..
        IF i_after > 0 THEN 
            SELECT seqid INTO v_tmp FROM _TREE_ WHERE id = i_after AND parent_id = i_parent LIMIT 1;
            IF v_tmp > -1 THEN 
                SET v_seqid = v_tmp+1;
            END IF;
        END IF;
            
        
        
        INSERT INTO _TREE_ SET 
            depth = v_depth , 
            seqid = v_seqid , 
            parent_id =  i_parent,
            name = i_name;
        
        
        SELECT LAST_INSERT_ID() INTO v_ret;
        
        #// fix the seqend id 
        
        
        UPDATE   _TREE_ SET seqid = seqid +1   WHERE seqid >= v_seqid  AND id != v_ret;
        
        RETURN v_ret;
        
    END $$
DELIMITER ;

You can now create a generic tree code by replacing my _TREE_ word with your table name:
eg.

#sed -e "s/_TREE_/yourtable/g" tree.js | mysql yourdb -f

Once all that is done the SQL to create tree nodes is very simple:

 delete from nametree; alter table nametree AUTO_INCREMENT =1 ;
 
 select nametree_add_node(0,0, 'one at top'); 
 call nametree_resequence(':');

 select nametree_add_node(1,0, 'two on one'); 
 call nametree_resequence(':');

select nametree_add_node(1,0, 'three on one (first)'); 
 call nametree_resequence(':');

 select nametree_add_node(1,2, 'four on one (last)'); 
 call nametree_resequence(':');

 select nametree_add_node(0,1, 'five on top (last)');
call nametree_resequence(':');

 select nametree_add_node(0,0, 'six is first'); 
 call nametree_resequence(':');

 SELECT * FROM nametree ORDER BY seqid;

Gives you a nice little table


+----+-----------+-------+-------+------+----------------------+---------------------------------+
| id | parent_id | seqid | depth | leaf | name                 | fullpath                        |
+----+-----------+-------+-------+------+----------------------+---------------------------------+
|  6 |         0 |     0 |     0 |    1 | six is first         | six is first                    | 
|  1 |         0 |     1 |     0 |    0 | one at top           | one at top                      | 
|  3 |         1 |     2 |     1 |    1 | three on one (first) | one at top:three on one (first) | 
|  2 |         1 |     3 |     1 |    1 | two on one           | one at top:two on one           | 
|  4 |         1 |     4 |     1 |    1 | four on one (last)   | one at top:four on one (last)   | 
|  5 |         0 |     5 |     0 |    1 | five on top (last)   | five on top (last)              | 
+----+-----------+-------+-------+------+----------------------+---------------------------------+

Enjoy tree.js (I name my mysql stored procs as js, just so my editor works better with them)

2008 roundup - Recession kicking in?

2008-12-22 13:05:00

Article originally from rooJSolutions blog

As my thumbs are twiddling waiting to see if projects start coming in next year, I thought I'd do a roundup of this years projects (open and closed). I was pondering if everyone else was seeing the effects of the recession, so I'd be interested to hear if you've also seen work slacking off.

Let's hope next year we see some more life fromthe Sales and Marketing, as my current reliance on word of mouth, may not keep feeding my mouth at this rate.

Anyway Merry Christmas and Happy new year to all.

Read on for my Year of Projects